Graphical Models for Driver Behavior Recognition in a SmartCar

In this paper we describe our SmartCar testbed: a realtime data acquisition system and a machine learning framework for modeling and recognizing driver maneuvers at a tactical level, with special emphasis on how the context a ects the driver's performance. The perceptual input is multi-modal: four video signals capture the contextual tra c, the driver's head and the driver's viewpoint; and a real-time data acquisition system records the car's brake, gear, steering wheel angle, speed and acceleration throttle signals. Over 70 drivers have driven the SmartCar for 1.25 hours in the greater Boston area. Graphical models, HMMs and potentially extensions (CHMMs), have been trained using the experimental driving data to create models of seven di erent driver maneuvers: passing, changing lanes right and left, turning right and left, starting and stopping. We show that, on average, the predictive power of our models is of 1 second before the maneuver starts taking place. Therefore, these models would be essential to facilitate operating mode transitions between driver and driver assistance systems, to prevent potential dangerous situations and to create more realistic automated cars in car simulators.